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Program overview 


The MacPhail Center for Music Online School Partnership Program aims to increase access 
to high-quality music education for students in rural schools in order to improve student 
interest and knowledge in music, and to improve or maintain positive school engagement and 
non-academic skills. The MacPhail Teaching Artists work collaboratively with music educators 
in schools to develop curriculum, implement the Online School Partnership, and work with 
students individually, in sectionals, or in large group settings on their music lessons via the 
internet. Teaching Artists teach classes at the partner schools remotely and in real-time through 
teleconferencing technology. Classes are typically 30-60 minutes in length, with a MacPhail 
Teaching Artist leading sessions on any of the following: current repertoire the students are 
working on in their ensemble; selecting and working on repertoire for a student to prepare 
for an audition; general tone, technique, and maintenance of instrument, etc. All of the 
session content is based on what the partner schools’ music educator would like the 
Teaching Artist to cover — the Teaching Artist then creates the session structure. 


The Online School Partnership has been providing lessons and working with schools outside 
the Twin Cities metro region since 2011. It currently has 27 Teaching Artists, partners with 
22 schools, and serves more than 800 students. All schools in the partnership had between 
six and 451 hours of instruction, totaling 1,800 hours of instruction. This evaluation, 
however, only focuses on eight schools who chose to participate in the evaluation: Yellow 
Medicine East High School, Willmar High School, Tri-Country Public Schools, Pipestone 
High School, MACCRAY High School, Kerkhoven-Murdock-Sunburg High School, 
Kingsland High School, and Detroit Lakes High School. Fourteen Teaching Artists from 
MacPhail were involved in the Online School Partnership. This evaluation focuses on 
individual and small group lessons with a MacPhail Teaching Artist. The types of lessons 
provided by the Online School Partnership are described as follows: 


= Group lessons: These lessons are ongoing and built on method books, handouts, and 
sequential practice planning. The MacPhail Teaching Artist consults with the partnering 
music educator to create ongoing assignments that encourage musicianship and solid 
playing techniques for the group of students, with a focus on one particular instrument 
or voice type. 


= Sectionals: In these lessons, MacPhail Teaching Artists work to boost the skills of 
specific sections of choir, band, or jazz bands. They work on repertoire the participating 
students are playing in large group. These students work on development of sound, 
rhythmic precision, general musical understanding, and performance skills. 


MacPhail Online School Partnership 1 Wilder Research, September 2018 
2017-18 Evaluation Results 


= Individual lessons: These lessons offer one-on-one instruction with an instrument or 
vocal specialist to work on general tone and technique. Teaching Artists may also 
provide support as a student prepares for all-state band or choir and contest preparation. 


Eighty-eight students took lessons through the Online School Partnership at the schools 
involved in this evaluation. These schools had between 12 to 451 hours of instruction, 
totaling 1,151 hours of instruction throughout the 2017-18 school year. 


Evaluation 


Wilder Research worked with the Online School Partnership to develop an evaluation design 
for the program. The evaluation includes implementation and outcome components, and 
gathers qualitative and quantitative data to answer the following research questions. (The 
Online School Partnership developed a logic model and these research questions reflect 
program activities and outcomes. See Appendix). 


Implementation evaluation key questions 


= How well is the Online School Partnership program being implemented? What are 
aspects of the program, if any, that need to be adjusted over time? 


= How well does the collaboration between MacPhail and the schools function, and 
how can it be strengthened? 


Outcome evaluation key questions 


= To what extent does student participation in the Online School Partnership program 
affect their interest, motivation, knowledge, and skills in music? 


= To what extent does student participation in the Online School Partnership program 
affect their attitudes about school (school motivation and engagement) and student 
non-academic skills? 


= To what extent does music educator participation in the Online School Partnership 
program affect their growth as an educator, their job satisfaction, and ability to handle 
many aspects of their work assignment? 
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Methods 


MacPhail and Wilder Research staff worked collaboratively to complete this evaluation. Both 
MacPhail and Wilder staff completed data collection responsibilities, and Wilder Research 
staff completed data entry, data analysis, and reporting. 


The following are the data collection instruments used in the evaluation. 


= Teaching Artist survey: In spring 2018, Wilder conducted a web survey with the 
MacPhail Teaching Artists. This survey asked about the overall vision and goals of 
the program, how collaboration with the partner schools has been going and how to 
improve it, successes of the Online School Partnership, and areas for improvement. 
Twelve of 20 Teaching Artists completed the survey, for a response rate of 60 percent. 


= Interviews with school music educators: Wilder conducted a phone interview with 
the school music educators involved with the Online School Partnership. In these 
interviews, educators were asked about the value and successes of the program, 
collaboration with MacPhail, and suggestions for improvement. All eight educators 
identified were interviewed for this evaluation. 


= Student survey: This survey was developed to measure student satisfaction with the 
Online School Partnership, perceptions of the lessons provided, and interest in music 
education and school generally. In spring 2018, 36 sixth- through twelfth-graders 
completed a survey, representing 41 percent of the 88 students in the program at these 
schools. The majority (89%) of students received individual lessons through the Online 
School Partnership, 22 percent took Online School Partnership lessons as a sectional, and 
3 percent took them as a class. 


= Music skills assessments: At the beginning and end of students’ time with the Online 
School Partnership, the MacPhail Teaching Artist they worked with completed skills 
assessments, looking at a variety of components of students’ performance abilities (tone 
quality, intonation, rhythm, etc.). Matched music skill assessments were done before 
and after programming for 77 of 88 students (88%) involved in the Online School 
Partnership from Detroit Lakes High School, Kerkhoven-Murdock-Sunburg High School, 
Willmar High School, MACCRAY High School, Kingsland High School, Pipestone 
High School, Yellow Medicine East High School, and Tri-County Public Schools. 
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Results 


This section describes the implementation and outcome results. Results are reported across 
the schools, and separately when appropriate. Results should be interpreted with caution 
given the small sample size. 


Implementation 


Successes 


Students’ musical skills improved with access to high-quality instruction; students 
enjoy their lessons and working with Teaching Artists. 


When asked about the biggest successes of the Online School Partnership, most commonly, 
music educators pointed to students’ skill improvement, with four of the eight school music 
educators mentioning this in the interview; relatedly, one instructor noted that the Online 
School Partnership led to his school having an award-winning band. Three music educators 
said that having access to other instructors was the biggest success, as it allows students 
access to high quality instruction and more instruction than they would otherwise receive. 
Beyond this, music instructors stated that the program encourages students to practice, 
prepares them for college band auditions, taps in to technology well, and provides access 
to rural and low-income students in Minnesota. 


Teaching Artists also commonly noted students’ skill improvement when asked this question, 
with six commenting on this. Five spoke about increasing access to high-quality music 
instruction in greater Minnesota. Beyond this, one Teaching Artist said that the program 
musical worlds” are 


2 66 


invigorates existing music programs, and one said that students 
expanded by providing them the resources they need to audition for all-state or honor bands. 


Music educators were asked about the potential value of the Online School Partnership. They 
spoke in varying ways about what they hope to see in their own programs with the help 
of the Online School Partnership. Most commonly, music educators mentioned the strong 
musicians coming out of the Online School Partnership, which will continue to influence 
their bands in years to come. Two educators spoke about growing the size of the program 
in terms of how many students are involved and taking lessons. Two wanted to incorporate 
more students within each lesson, either by having Online School Partnership lessons 
provided for sectionals or for the full band. Music educators also stated that the program 
will continue to add capacity to their departments and will continue to be a resource for 
band students. All music educators indicated that they plan to continue with the program, 
as long as funding permits. 
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Teaching Artists were also asked what the potential value of the program is. Nine Teaching 
Artists spoke about how the program increases access to high-quality music instruction in 
greater Minnesota, where this instruction is otherwise not available. Two of these Teaching 
Artists specifically spoke about leveling the playing field between rural and urban music 
students. Six of the twelve Teaching Artists noted that students can benefit from expert 
instruction from a musician specifically trained on their instrument. 


Students were satisfied with the Online School Partnership program. Students were asked 
to what degree they agree with the statements “I like coming to the online sessions” and 
“T would recommend these sessions to other students.” Ninety-six percent of students stated 
that they “agree” or “strongly agree” that they like coming to the online sessions, and all 
students reported that they would recommend the sessions to others (Figure 6). 


In terms of satisfaction with the technology used for the online sessions, 93 percent agreed 
or strongly agreed with the statement “the technology used for these sessions made it easy 
to learn” (Figure 7). 


Students were also positive about the Teaching Artist who led their online sessions; all 
students said they liked their Teaching Artist (Figure 8). Students also said their Teaching 
Artist set clear expectations for practicing and learning at home (96% agree or strongly 
agree) and used class time well (92% agreed or strongly agreed). 


When asked what they liked most about the online sessions, students most frequently said 
that they enjoyed that they gained new skills, techniques, or musical knowledge (45%) and 
that they appreciate their time with and feedback from their Teaching Artist (30%; Figure 9). 


Collaborative efforts aid student learning and allow for positive working relationships 
between students and Teaching Artists. 


In the interview, the music educators were asked to describe the collaborative efforts between 
their school and MacPhail. Three of the eight music educators said that MacPhail staff are 
easy to work with, specifically noting that they were flexible, supportive, and organized. 
One music educator pointed out how considerate and understanding MacPhail staff are 
regarding changes in the schedule or plan. Three music educators called out the physical 
visits from Teaching Artists as particularly collaborative and helpful; one music educator 
spoke positively about the opportunity their students had to visit the MacPhail Center and 
meet with their Teaching Artists in person. Two music educators said that the Online School 
Partnership allows for students to form great working relationships with their Teaching 
Artists, which can be deeply meaningful for students. Beyond this, one music educator said 
that having specialists in a student’s instrument has been beneficial, and another music 
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educator said that it is helpful for them to have the messages they send to their students 
around specific techniques or approaches amplified by the Teaching Artist. 


Teaching Artists were asked if the collaboration has been successful between MacPhail 
and the partner schools. All 12 responded positively, with some adding a caveat that they 
only know their own experience. Most commonly (10 of 12), Teaching Artists pointed to 
students’ progress in their musical skills as evidence that the collaboration was effective; 
another Teaching Artist said that the overall level of the full band at a partner school has 
improved. Two Teaching Artists mentioned students who had gotten into all-state band in 
part because of their lessons through the program. Three Teaching Artists spoke about the 
strong working relationships they have forged with students, with one noting that it was an 
experience in mentorship. Two others mentioned the emotional benefits to students, with 
one saying that it is clearly an enjoyable experience for students, and the other noting that 
student confidence has grown. Beyond this, one Teaching Artist mentioned that because 
the Online School Partnership reviews fundamental skills and techniques, music educators 
can be more flexible in their music selection, and one Teaching Artist noted how well-run 
the program is managed by MacPhail staff. 


Opportunities for improvement 


Schools struggle with their own technological capacity. 


When asked what the primary areas for improvement are for the Online School Partnership, 
half of the music educators (4 of 8) said that technology improvements are most important; 
however, all three noted that it was their own schools that needed to improve their 
technological capacity, and that MacPhail had done all they could to ensure this. Four music 
educators did not have suggestions for MacPhail, and that they were very pleased with the 
program, with one music educator who was appreciative of the ease of rescheduling lessons 
and that the technology works well. One music educator said it’s essential that the program 
continue to be subsidized, because their program would not be able to continue in the Online 
School Partnership otherwise. Two music educators made specific notes about areas for 
improvement. One noted that the logistics of scheduling and facilitating lessons could be 
made easier, preferring that Teaching Artists be able to work directly with students. Another 
music educator noted that it would be helpful if the Teaching Artists could make the amount 
of feedback they provide more consistent. This music educator noted that while he gets 
frequent updates about student progress from some Teaching Artists, he doesn’t hear 
anything from others. 
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When students were asked to suggest program improvements, they most frequently said 
that the technology could be improved (38%; Figure 10). Beyond this, students most 
commonly suggested that sessions be provided more frequently (14%), that students and 
their Teaching Artist meet in person more (10%), and that the sessions be longer (2 responses; 
7%). Two responses (7%) were suggestions related to music; to increase the amount of 
music worked on in lessons, and to work on the same music as the full band. “Other” 
responses (14%) were to improve time management in the lessons, to assign more at-home 
work, to focus on more areas within the lessons, and to increase the amount of structure 
in the lessons. Four respondents (14%) said they had no suggestions. 


Similar to music educators and students, Teaching Artists most commonly identified 
technology as an area for improvement, with three of 12 Teaching Artists noting this. Two 
Teaching Artists suggested expanding the program to include more students, more schools, 
and more areas of the state. Other suggestions included finding opportunities for students 
to visit the MacPhail Center more often, to establish a staff position at MacPhail to assist 
Teaching Artists and music educators with schedule coordination, and to ensure that music 
educators are enthusiastic about their engagement in the program. Three Teaching Artists 
said they had nothing to recommend. 


Music educators and Teaching Artists are pleased with the collaboration that exists 
between MacPhail and participating schools. 


When asked about recommendations music educators had to improve the collaboration, 
three stated that they had no suggestions for improvement, and one said that they especially 
appreciate the current level of communication they have with MacPhail staff. Two music 
educators said that technology issues continue to be the main hindrance to the program, 
but that these issues were on their school’s end and that MacPhail could not do anything 
to make it better. No common themes emerged among suggestions from the music educators. 
Recommendations given were to increase the consistency of feedback from Teaching Artists 
to music educators, to add another Teaching Artist visit to music educator’s schools, to 
add to the current set of video tutorials and ensure that educators outside of the Online 
School Partnership are aware of them, and to ensure that Teaching Artists’ expectations 
of students are aligned with the student’s capacity. 
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Teaching Artists were asked to make recommendations to improve the partnership. Most 
commonly (3 of 12 responses), Teaching Artists said that the schools’ technology could 
be improved. Two Teaching Artists noted that it is highly beneficial for them to physically 
visit the schools and their students, and would like additional funding to allow for more 
visits. A couple of Teaching Artists commented on areas where additional funding would 
be helpful; one said they would like to have a planning session between music educators 
and Teaching Artists prior to any lessons beginning, and another suggested expanding the 
program in schools. Other suggestions include MacPhail establishing a digital music library 
or music webpage to allow for easier access to resources, providing longer sessions, having 
more emphasis on students practicing outside of school and during the summer, and keeping 
lesson group sizes manageable. Two Teaching Artists said they had nothing to recommend. 


Outcomes 


Benefits to students in music 


Students gain musical skills and knowledge 


When students were asked to how much they agree with the statement “I learned new 
musical skills that helped me in band/choir,” students were generally positive; 97 percent 
either “agree” or “strongly agree” with the statement (Figure 11). 


Teaching Artists assessed students’ musical skills at the beginning and end of their time 
with the Online School Partnership. Students were assessed in a variety of areas: tone quality, 
intonation, rhythm, technique, musical expression, execution (for percussion students), 
articulation (for wind students), and music reading skills (Figures 12-19). 


Overall, students showed moderate growth in most skills. Students showed the most growth 
in musical expression, with a mean improvement of 1.1 points (where 1=poor, 2=fair, 
3=good, 4=excellent, and 5=superior). 
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1. Musical skill growth 


Musical expression ae 1.1 
a 0.2 
DN os 
De 0.3 
Articulation = 0.8 
Tone quality J 0.8 
Reading skills [I 0.7 
WH os 


Note. This chart shows change in mean scores for all Online School Partnership participants included in this evaluation from 
pre- to post- skills assessments. 1=poor, 2=fair, 3=good, 4=excellent, and 5=superior. 


Execution 


Technique 
Rhythm 


Intonation 


Students were rated highest in execution, rhythm, and musical expression, with a mean score 
of 3.9 in each in their post assessments. 


2. Mean scores at end of lessons 


Musical expression 
Execution 


Rhythm 


Technique TTT 
Tone quality TT >: 


Articulation 


Reading skills 


Intonation 


Note. This chart shows mean scores at the end of lessons for all Online School Partnership participants included in this 
evaluation from pre- to post- skills assessments. 1=poor, 2=fair, 3=good, 4=excellent, and 5=superior. 
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The music skills assessment also asked Teaching Artists to provide ratings in a student’s 
musical confidence and ability to focus (Figures 20 and 21). Students made improvements 
in mean scores of 0.8 points in confidence (from mean scores of 3.5 to 4.3) and 0.2 points 
in ability to focus (from mean scores of 4.2 to 4.4). 


3. Musical attitude 


ns +: 
Ability to focus 
Ss “2 


Confidence <—— -:: 
ss; 


=Post m=Pre 


Note. This chart shows mean scores at the end of lessons for all Online School Partnership participants included in this 
evaluation from pre- to post- skills assessments. 1=poor, 2=fair, 3=good, 4=excellent, and 5=superior. 


Students were asked if they participated in any musical competitions in the previous school 
year. 83 percent of respondents across schools had done so (Figure 24). All students who 
participated in a competition participated in solo and ensemble, 21 percent participated in 
honor band, and 10 percent participated in all-state (Figure 25). 


4. Competition participation 


100% 


21% 
10% 
=) 


All-State Honor Solo and 
Band Ensemble 


Note. This chart reflects which competitions students participated in, of students who indicated that they had participated in a 
competition. Students were allowed to select multiple responses. 


In interviews, all eight music educators indicated that the Online School Partnership has 
increased student skill and knowledge, with all music educators specifically noting that it 
is helpful to have students learn from a specialist in their instrument and learn techniques 
specific to their instruments. Two music educators said that participating in the lessons 
increases students’ repertoire, one noted that a student was able to work on their college 
auditions during the lessons, and one noted that a student was able to increase their 
confidence, which improved their playing. 
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Students show interest and motivation in music 


Ninety-seven percent of students reported that they “agree” or “strongly agree” that coming 
to the online sessions increased their appreciation of music (Figure 22). Students were 
interested in continuing to develop their musical skills (97%) and were more motivated to 
do well on their instruments (96%). Looking to the future, a little less than one-third of 
students (29%) stated that they are interested in pursuing a college degree in music. 


5. Motivation 


|work hard in school. [ii 100% 
| am interested in continuing to | 97% 
develop my musical skills. 
Coming to the online sessions ees 97% 
increased my appreciation of music. 
Aelon ty incncrt, iis oc, 
well on my instrument. 


| am interested in pursuing a college degree 
in music after graduating from high school. oo 29% 


Note. This chart shows the percentage of students reporting “agree” or “strongly agree” to these questions. 


To assess their motivation to continue developing their musical skills, students were asked 
if they wanted to continue in the MacPhail program the following year. Eighty-four percent 
of all students wanted to continue in the program (Figure 22). Students who said that they 
“disagree” (13%) or “strongly disagree” (4%) with continuing the program indicated that 
they were graduating or unsure of their preferences. 
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Benefits to students in school 


Students have positive attitudes toward school 


Students reported that they work hard in school, with all students either agreeing or strongly 
agreeing that they do so (Figure 23). 


In interviews, music educators were hesitant to say that the Online School Partnership was 
improving students’ general attitudes about school. Four music educators noted that the 
students who enroll in lessons through the Online School Partnership already love school 
and that the lessons don’t cause a change in attitude, and another noted that the participating 
students just see the Online School Partnership as an extension of the school, rather than 
something new. Three music educators thought it made students feel more positive about 
school, either by giving more validity to the band they play in, giving them confidence, or 
by giving them something to look forward to when they come to school. 


Benefits to music educators and schools 


Music educators grow as teachers, and have increased resources and capacity to grow 
their music programs 


All four music educators were very positive about their own growth and learning due to 
participation in the program. Three music educators spoke about how students going through 
lessons are able to pass along what they’ve learned to their music educator and throughout 
their band. They mentioned being able to go to the Teaching Artist with any questions, which 
was particularly helpful due to the lack of peers at their school or in their general geographical 
region. Two music educators said that it increases their own capacity to keep things running 
smoothly in their classes or in their program. Two music educators said they’ve benefited from 
hearing from the Teaching Artist directly, either by sitting in on teleconferenced lessons or 
through the physical visits. Additionally, each music educator said that they’ve gotten ideas 
for music their students can use for auditions and that it’s helpful to work together with a 
Teaching Artist on specific skills they would like a student to develop. 


All educators were positive about the impact the Online School Partnership is having on 
their program. Four music educators spoke about the improvements the students have seen 
in their musical skills; another two music educators said that students’ acceptance into 
all-state band was due to participation in the program. Another music educator noted that 
the program increased students’ interest in music. Two music educators spoke about the 
benefit of increased capacity, where they could increase the number of students getting 
individual lessons or being able to use their time in more efficient ways, and one music 
educator noted that the program increased access to music education for those who could 
not or would not want to travel to the Twin Cities for weekly music lessons. 
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Four music educators spoke about improvements regarding emotion and relationships around 
music. Two music educators said that participation in the program allowed high-achieving 
students to push themselves to another level in both their interest in pursuing music and 
their own capacity to do so, including auditioning for college bands. Music educators 
described improvements in students’ confidence, focus, and leadership, and also mentioned 
that students form good working relationships with Teaching Artists. 


Issues to consider 


The Online School Partnership data indicate that participants are doing well in the program. 
Students generally enjoy their classes and are actively engaged and progressing in their 
music education. Similarly, music educators are positive about their satisfaction with the 
program and feel it is a helpful resource. 


Based on the results available to date, the following are a few points to consider in 
future program. 


MacPhail program staff should continue to: 

= Be flexible and attentive to the partner schools’ needs; music educators noted how helpful 
this is to their work. 

m Provide avenues of financial support to partner schools; music educators said that this 
was crucial to their continued participation in the program. 

Going forward, MacPhail could consider the following improvements: 

= Help schools pay for technology improvements; technology issues were the most 
commonly cited issue with the Online School Partnership. 


=m Secure additional funds to allow the program to grow to reach more students, schools, 
and areas of the state, as both music educators and Teaching Artists expressed interest 
in this area. 


m= Secure additional funds for Teaching Artists to visit the schools; students, music 
educators, and Teaching Artists all noted that this was one of the best components of 
the program and would like more in-person visits. 


m= Advocate at the state level for greater access to high-speed internet in rural areas. 
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Figures 


Implementation 


6. Students’ general satisfaction with the Online School Partnership 


Strongly Strongly 
disagree Disagree Agree agree 
School N 1 2 3 4 
| like coming to the online sessions. Allschools 28 0% 4% 50% 46% 
| would recommend these sessions to Allschools 28 0% 0% 39% 61% 
other students. 
7. Students’ satisfaction with technology used 
Strongly Strongly 
disagree Disagree Agree agree 
School N 1 2 3 4 
The technology used for these sessions Allschools 28 0% 7% 54% 39% 
made it easy to learn. 
8. Students’ report of instructor 
Strongly Strongly 
disagree Disagree Agree agree 
School N 1 2 3 4 
| liked the Teaching Artist who ledthe class. Allschools 28 0% 0% 14% 86% 
The Teaching Artist used class time well. Allschools 27 0% 7% 33% 59% 
The Teaching Artist set clear expectations Allschools 28 0% 4% 46% 50% 
for practicing and learning at home. 
Note. Totals may not equal 100 due to rounding 
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9. Student satisfaction with Online School Partnership classes 


What did you like most about this class? N=33 N Percent 
New skills/technique/knowledge 15 45% 
Feedback/time with Teaching Artist 10 30% 
Dedicated time to work on music 6 18% 
Teaching Artist was kind/fun/nice 4 12% 
Teaching Artist was skilled/good at teaching 4 12% 
Other 3 9% 


Notes. Students may have noted more than one thing they liked most about the class, resulting in a total >100%. Percentages 
given reflect the number of times a response was given divided by the total number of responses. 

“Other” responses include positive comments about the accessibility of the lessons, flow of the lessons, and the 
personalization of the lessons. 


10. Student suggestions for program improvement 


What would you improve? N=29 N Percent 
Technology improvements 11 38% 
More frequent sessions 4 14% 
No suggestions 4 14% 
More physical visits 3 10% 
Longer sessions 2 7% 
Music suggestion 2 7% 


Other 4 14% 


Notes. Students may have noted more than one thing they liked most about the class, resulting in a total >100%. Percentages 
given reflect the number of times a response was given divided by the total number of responses. 

“Other” responses include student requests for more homework, better time management of lessons, and request for more 
content areas to be covered in lessons 


Outcomes 


11. Students’ report of musical skills and knowledge 


Strongly Strongly 
disagree Disagree Agree agree 
School N 1 2 3 4 
| learned new musical skills that helped Allschools 27 0% 4% 30% 67% 
me in band/choir. 
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12. Tone quality before and after online sessions 


Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 76 15% 58% 24% 4% 0% 3.8 
All schools +0.8 
Pre 76 1% 21% 53% 24% 1% 3.0 
13. Intonation before and after online sessions 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 66 11% 53% 32% 5% 0% 3.7 
All schools +0.5 
Pre 66 0% 38% 47% 14% 2% 3.2 
14. Rhythm before and after online sessions 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 74 23% 47% 26% 4% 0% 3.9 
All schools +0.8 
Pre 74 5% 22% 51% 20% 1% 3.1 
15. Technique before and after online sessions 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 75 15% 53% 28% 4% 0% 3.8 
All schools +0.9 
Pre 75 1% 20% 48% 27% 4% 2.9 
16. Musical expression before and after online sessions 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 74 19% 54% 22% 5% 0% 3.9 
All schools +1.1 
Pre 74 5% 14% 43% 34% 4% 2.8 
17. Execution (Percussion) 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 73 16% 56% 26% 1% 0% 3.9 
All schools +0.9 
Pre 73 4% 21% 49% 26% 0% 3.0 
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18. Articulation (Winds) 


Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 76 13% 50% 34% 3% 0% 3.7 
All schools +0.8 
Pre 76 1% 18% 49% 32% 0% 2.9 
19. Reading skills 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 75 13% 52% 29% 5% 0% 3.7 
All schools +0.7 
Pre 75 1% 27% 41% 28% 3% 3.0 
20. Confidence 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 75 36% 56% 7% 1% 0% 4.3 
All schools +0.8 
Pre 75 17% 33% 35% 13% 1% 3.5 
21. Ability to Focus 
Superior Excellent Good Fair Poor 
School N 5 4 3 2 1 Mean Change 
Post 76 55% 32% 11% 3% 0% 4.4 
All schools +0.2 
Pre 76 47% 26% 21% 5% 0% 4.2 
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22. Students’ report of musical interest 


Strongly Strongly 
disagree Disagree Agree agree 
School N 1 2 3 4 
Coming to the online sessions increased Allschools 27 4% 0% 56% 41% 
my appreciation of music. 
| am interested in continuing to develop Allschools 28 0% 4% 43% 54% 
my musical skills. 
| am interested in pursuing acollege degree Allschools 28 11% 61% 18% 11% 
in music after graduating from high school. 
| am more motivated to do well on my Allschools 28 0% 4% 39% 57% 
instrument 
| am interested in continuing with the Allschools 24 4% 13% 38% 46% 
MacPhail Online Partnership program 
23. Students’ report of work in school 
Strongly Strongly 
disagree Disagree Agree agree 
School N 1 2 3 4 
| work hard in school. Allschools 27 0% 0% 22% 78% 


Notes. Table draws on information from the Student Survey. Totals may not equal 100 due to rounding 


24. Student participation in competition 


School N Yes 


All schools 35 83% 


25. Competitions participated in 


School N _Solo/Ensemble_ _ Honor Band All-State 
Allschools 29 100% 21% 10% 
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Appendix 


Open-ended comments 


A1. Student survey open-ended comments: “What did you like most about this class?” 


All Schools (N=33) 
Increased musical skill and knowledge (N=17) 


Learning more playing skills. 

They were very informative. 

Learning so much more about my instrument and music in general. 

| liked how easy it was to connect with [instructor] while learning how to better my skills on flute. 

Learning and working on new things with someone | enjoy working with. 

Working on my concert music in depth. 

| enjoyed the practice at what | enjoy doing which is percussion. 

Learning more advanced skills. 

The opportunity to improve my skills. 

| liked getting new feedback to improve me musical ability. 

It really helped me widen my horizon and look at different music. 

How much | learned. 

Playing mew music and learning new notes. 

| liked learning how to play better. 

Learning new things in different types of music and how to make it better. 

Learning new ways to improve my skill playing the trombone. 

| really enjoyed getting to learn new ways to improve my ability to play my solo. 
Support, instruction, and encouragement (N=10) 


What | liked most about the online sessions is that it gave me more time to work individually with an 
instructor. 


The face-to-face time. 

The one-on-one time that was spent on improving my musical skill. 

| loved the personal one-on-one time. 

The time that was given to me as an individual musician. 

Devotion of practice to work one on one with a very skilled instructor. 
| liked getting one-on-one instruction. 

One-on-one help. 

It was more personalized, the instructor played the same instrument. 


How much my teacher pushed me to get better. 


MacPhail Online School Partnership 19 Wilder Research, September 2018 
2017-18 Evaluation Results 


A1. Student survey open-ended comments: “What did you like most about this class?” 
(continued) 


All Schools (N=35) 
Other (N=8) 


Working on my solo. 


It was fun interacting with someone who understood my instrument and could offer me advice on how to 
advance myself with it. 


We got a lot done with the time we had. 

[Instructor] was funny and pushed me to my ability, but [the instructor] wasn't extremely hard on me. 
My instructor was very nice. 

It set aside time to work on solos and other music besides my concert band music. 

Accessibility of lessons. 

The flow of the lessons. 


A2. Student survey open-ended comments: “What would you improve about 
this class?” 


All Schools (N=25) 
Technical Issues (6) 


Better video quality. 
The only problems | have is with internet connection so if there is a way to improve that, that would be nice. 
Sound quality. 
The functionality of Skype during our sessions. 
Skype connections. 
| would improve the internet connection so we don’t have any delays in the lesson. 
Nothing (6) 
Nothing (5) 
Nothing, the sessions were great. 
Longer/More Frequent classes (4) 
| would either add more time to it or make it more often. 
| would make them longer. 
Maybe make them a little more frequent. 
Have them more often. 
Meeting Instructor (3) 
| think it would be fun to meet the instructor in person more often. 
Perhaps an in person check point sometime during the year. 
Meeting in person. 
Note. Responses might have been broken up and arranged into their respective categories. 
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A2. Student survey open-ended comments: “What would you improve about 
this class?” (continued) 


All Schools (N=25) 
Other (7) 


A more structured lesson. 

| would improve the amount of music and content learned throughout the lessons. 
Focusing on more than one thing during a session. 

Work some with the music being used in class. 

Time management. 

Work more at home. 

Practicing more often. 


Note. Responses might have been broken up and arranged into their respective categories. 
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Logic model 


MacPhail Online School Partnership Logic Model 


Overview: The MacPhail Online School Partnership was established to partner with schools in greater Minnesota to support their music program’s 
goals by providing MacPhail faculty to teach students remotely. By teaching classes remotely, MacPhail aims to increase students’ skills, knowledge, 
and interest in music, and provide the resources for music educators to grow professionally. 


EL > > OUTCONES 


Teaching artists 


Videoconferencing 
technology 


MacPhail funding 


Other funding 


Facilities and 
physical space 


Students 


Music educators 


Teaching artists work with music educators 


to determine curriculum for students 


SHORT TERM 


LONG TERM 


Tailored lessons are offered 
to students in addition to their regular 
music classes, which include: 
- Group lessons 
- Sectionals 
- Individual lessons 


Student outcomes 


- Students show improved musical skill and 
knowledge!2:34 


- Students show increased interest, appreciation, 
and motivation in music® 


- Students show improved attitudes toward school? 
- Students show growth in non-academic skills? 


Students continue their 
music education 
throughout high school 
and in college? 


Student musical performances 


Music educator outcomes 
- Music educators grow as teachers* 


- Music educators have increased resources and 
capacity to grow their music programs* 


Rural students are more 
likely to have the 
opportunity to perform 
and compete at a high 
levels 


Music educators observe Teaching Artists 
and learn of new teaching approaches, 
methods, and resources 


Evaluation and documentation 
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MacPhail outcomes 


- MacPhail is able to reach out to students who would 
otherwise have no access to the organization® 


- MacPhail creates partnerships with schools 
throughout the state*® 


‘Measured by music skills assessment 
2 Measured by progress reports 
3 Measured by student survey 
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Partner schools 
throughout the state are 
better able to offer high- 
quality music education 
to their students® 


4Measured by music educator survey 
5 Measured by Teaching Artist survey 
8 Measured by program records 


Survey instruments 


MacPhail MacPhail Teaching Artist: 


CENTER FOR MUSIC 
Student(s): 

School: 

Director: 

Instrument({s): 


Please place one of these numbers in each box below; then total. 5 = superior; 4 = excellent; 3= good; 2=fair; 1=poor 


Performance Factors Comments 


Tone Quality 
Clarity, consistency, control, 
focus, warmth 


Intonation 
Accuracy, consistency of pitch 


Rhythm 
Accuracy of note and rest 
values, correctness of meters, 
duration, pulse, steadiness 
Balance Blend 
Accompaniment, awareness 
of ensemble, likeness of 
qualities, matching of 
registers (if applicable) 
= 


Technique (facility/accuracy) 
Attacks, releases, control of 
ranges, mechanical skill 


Interpretation/Musicianship 
Dynamics, emotional 
involvement, artistry, 
phrasing, style, tempo 


Diction — Vocal 
Bowing — Strings 
Articulation — Winds 
Execution - Percussion 


Other Performance Factors 
General conduct, 
mannerisms, poise, posture, 
etc. 


TOTAL POINTS 


Signature of Teaching Artist: 


Date: __ 
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MacPhail Online School Partnership Student Survey 
Spring 2016 


Choir/Band Teacher: Today’s date: 


MacPhail Teaching Artist: Type of session (individual, sectional, or 
class): 
This survey will ask you questions about you and the music sessions you participated in this semester. We 
are interested in your honest thoughts and ideas, so there are no right or wrong answers. 


Your answers are important. Please indicate how much you agree with each of these statements and 
fill out all parts of the survey. 


Strongly Strongly Don’t 
disagree Disagree Agree agree Know 

1. | learned new musical skills that helped me in 

band/choir. LI CP LI sy Ls 
2. | like coming to the online sessions. LI L|2 L|s Ls Ls 

| liked the Teaching Artist who led the class. LI C1 L]s Ll Ls 
4. Coming to the online sessions increased my 

appreciation of music. LI L? Ls Lis LI 
5. The Teaching Artist used class time well. | [2 []s [_|4 [_]s 
6. The Teaching Artist set clear expectations for 

practicing and learning at home. LI LF LI sy Ls 
7. |am interested in continuing with the 

MacPhail Online Partnership program. Lh L? Ls Lis Li 
8. lam interested in continuing to develop my 

musical skills. Li Li Lis Li Li 
9. |am interested in pursuing a college degree in 

music after graduating from high school. Li Li Us Li Us 
10. |am more motivated to do well on my 

instrument. LI L Lh Lis Ls 
11. The technology used for these sessions made 

it easy to learn. LI L? Ls Lis Ls 
12. | would recommend these sessions to other 

students. LI L? Ll Li Ls 
13. | work hard in school. L} L|2 L]s L|s Ls 
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15. What did you like most about this class? 


16. What would you improve about this class? 


17. Did you participate in a competition this year? 
L]' No 
Ll? Yes 


If yes, check all that apply: 
L]*  Solo/Ensemble 

L]4 Honor Band/Choir 
Lis Al-State 


THANK YOU FOR COMPLETING THE SURVEY! 
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